Papers with thresholding-based sparsification
CHESS: Optimizing LLM Inference via Channel-Wise Thresholding and Selective Sparsification (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for activation sparsification do not capture the relationship between activation and model performance. |
| Approach: | They propose a general activation sparsification approach using channel-wise thresholding and selective sparsifying to capture the relationship between activation and model performance. |
| Outcome: | The proposed approach reduces the number of activated neurons during inference by 1.27x over eight downstream tasks while activating fewer parameters than existing methods. |